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Operational  Thread  Development 

A  Structured  Approach  to  Capability  Analysis 


Abstract 

This  paper  will  introduce  the  Operational  Thread  Development  (OTD)  methodology  for 
analyzing  warfighting  capability  and  assessing  the  contribution  of  potential  solutions  to  filling 
identified  capability  deficiencies.  The  concept  of  effects- driven,  capabilities-based  planning  and 
analysis  has  dominated  the  military  force  structure  planning  and  acquisition  cycles  in  recent 
years.  Recently  revised  documents  such  as  CJCSI  3170.01E,  Joint  Capabilities  and  Development 
System  require  a  rigorous  approach  to  analyzing  potential  solutions  to  warfighting  capability 
deficiencies.  However,  attempting  to  implement  this  guidance  in  a  large-scale  field  experiment 
has  proven  challenging. 

OTD  is  a  new,  structured  approach  to  planning,  designing,  and  analyzing  a  large-scale  field 
experiment  that  supports  the  capabilities-based  construct.  Specifically,  this  paper  will  describe 
the  framework  for  capability  analysis  that  was  used  for  planning  Joint  Expeditionary  Force 
Experiment  2006  (JEFX  06);  a  process  for  developing,  executing,  and  analyzing  operational 
threads;  and  a  web-based  toolset  that  supports  this  approach. 

Background 

JEFX  06  was  the  sixth  in  a  series  of  large-scale  Air  Force  experiments  designed  to  assist  the  US 
Air  Force  prepare  for  the  challenges  of  21st  Century  Expeditionary  Air  and  Space  Force 
operations.  To  that  end,  the  experiment  attempted  to  anticipate  and  model  a  future  command  and 
control  system,  based  on  capabilities  in  the  Space  and  Command,  Control,  Communications,  and 
Computer,  Intelligence,  Surveillance,  and  Reconnaissance  Concept  of  Operations  (S&C4ISR 
CONOPS).  Specifically,  this  experiment  addressed  four  broad  capability  goals: 

•  Continuous  Theater  Air  Planning  and  Dynamic  Execution  (CAPE):  Conduct 
continuous  theater  air  planning.  Provide  near-real-time  Situational  Awareness  (SA)  using 
data  links.  Assess  the  effects  of  kinetic  and  non-kinetic  actions  and  conduct  dynamic 
execution. 

•  Fusion  for  the  Air  and  Space  Operations  Center  (AOC):  Given  adequate  preparation 
of  the  battlespace,  fuse  data  and  information  from  multiple  sources  and  cross-security 
boundaries  to  rapidly  achieve  and  maintain  battlespace  awareness  supporting  both  kinetic 
and  non-kinetic  effects. 

•  ConstellationNet  (CN):  Command,  control,  defend,  and  manage  an  integrated  air,  space, 
and  terrestrial  network  to  include  airborne  Internet  Protocol  (IP)  networks.  Enable  near- 
real-time  Joint  Blue  Force  SA  and  Combat  Identification  for  airborne  and  mobile  ground 
forces  operating  in  hostile  territory. 

•  Homeland  Security  /  Homeland  Defense  (HLS/HLD):  Collect,  fuse,  and  disseminate 
information  in  coordination  with  joint  and  federal  agencies  in  support  of  HFS/HFD 
Operations.  Integrate  Agile  Combat  Support  (ACS)  Expeditionary  Site  Planning. 

As  the  managers  of  JEFX,  the  Air  Force  Experimentation  Office  (AFEO)  brought  together 
operators,  engineers  and  software  developers  to  generate  new  technology  and  develop  processes 


that  would  improve  operational-level  warfighting  capability  across  the  four  capability  goals. 
During  the  initial  planning,  these  capability  goals  were  further  focused  by  the  experiment 
planners  into  specific  capability  deficiencies  that  included  Measures  of  Success  (MOS)  to 
characterize  progress  in  achieving  the  capability  goals.  In  addition,  AFEO  employed  an 
incremental  development  approach  that  included  three  preliminary  “spiral”  events  as  precursors 
during  the  months  prior  to  the  main  experiment.  The  task  for  the  analysis  team  was  to  develop  a 
plan  for  assessing  the  contribution  of  new  technology,  processes,  and  organizational  structures  to 
these  capability  areas. 

Analysis  Framework 

The  concept  of  developing  “threads”  to  supporting  Air  Force  experimentation  originated  during 
JEFX  2004.  During  that  experiment,  the  Army  Close  Air  Support  and  Situational  Awareness 
(ACASSA)  initiative  was  developed  around  two  “seams”  and  five  supporting  threads.  The  seams 
were  analogous  to  capability  deficiencies,  and  the  threads  identified  unique  solutions  to  close 
those  seams.  Ultimately,  the  Air  Force  Command  and  Control  and  Intelligence,  Surveillance,  and 
Reconnaissance  Center  (AFC2ISRC)  AFEO  leadership  endorsed  this  model  as  a  basis  for  the 
planning  of  JEFX  06.  The  benefit  of  such  an  approach  was  generally  recognized,  although  there 
was  some  confusion  regarding  terminology,  especially  for  those  not  involved  in  the  daily 
planning  of  the  ACASSA  threads. 

Terminology 

Based  on  the  ACASSA  model,  and  as  a  result  of  leadership  guidance,  the  AFEO  Analysis  and 
Assessment  Branch  led  an  effort  to  develop  a  complete  and  consistent  framework  for 
Operational  Thread  Development  (OTD).  The  OTD  framework  included  a  standard  terminology 
(Appendix  1).  We  recognized  early  that  simply  having  a  glossary  of  terms  and  common 
understanding  was  insufficient  for  developing  a  framework,  process  and  web-based  toolset  that 
would  eventually  facilitate  the  large-scale  analysis  planning  that  typically  precedes 
experimentation.  As  such,  we  referenced  terms  from  joint  and  service  doctrine  and  also 
developed  a  taxonomy  of  terms  describing  logical  relationships  among  them,  as  depicted  in 
Figure  1. 
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Figure  1 — Analysis  Planning  Framework 


Capability-Based  Approach 

In  addition  to  standardizing  tenninology,  we  also  found  it  necessary  to  revise  our  previous 
planning  efforts  to  reflect  a  capabilities-based  approach  to  experimentation.  The  basic  challenge 
for  the  core  analysis  team  was  to  develop  an  approach  that  would  allow  for  a  rigorous 
examination  of  warfighting  capability,  despite  the  fact  that  the  very  term  “capability”  was  often 
mis-used  or  misunderstood.  In  fact,  our  basic  premise  was  that  a  capability  could  not  be  directly 
observed  and  measured. 

Because  a  capability  cannot  be  directly  observed  and  measured,  each  capability  deficiency  can 
be  examined  through  the  use  of  an  “operational  thread” — an  important  component  to  the  overall 
operational  assessment  conducted  during  JEFX  06.  An  operational  thread  is  a  series  of  related 
operational  tasks  that  are  specifically  focused  to  highlight  the  contribution  of  experimental 
initiatives  or  infrastructure  systems  to  an  Air  and  Space  Operations  Center  (AOC)  or  other  basic 
Command  and  Control  (C2)  process.  An  operational  thread  is  a  design  feature  of  the 
experiment,  allowing  experiment  planners  to  influence  player  activity  in  desired  areas.  These 
threads  generally  have  a  well-defined  beginning  and  end,  and  are  often  executed  within  a  single 
period  during  the  experiment.  Operational  threads  are  typically  stimulated  by  scenario  events 
from  the  Master  Scenario  Event  List  (MSEL)  but  they  may  involve  a  significant  degree  of  “free 
play”  by  the  players. 


The  JEFX  06  operational  threads  were  defined  and  prioritized  prior  to  the  second  and  third  spiral 
events,  based  on  a  complete  understanding  of  the  chosen  scenario,  capability  deficiencies, 
experiment  initiatives,  and  underlying  infrastructure.  Specific  scenario  events  were  developed 
based  on  the  identified  operational  threads.  During  previous  experiments  that  did  not 
consistently  employ  operational  threads,  MSELs  were  often  associated  with  experiment 
initiatives  without  a  clear  indication  of  the  desired  outcome.  Operational  threads  were  scheduled 
in  advance,  to  the  extent  possible,  and  related  directly  to  daily  experiment  objectives. 

Developing  experiment  objectives  and  operational  threads  prior  to  spiral  events  ensures  that 
assessment  and  experiment  objectives  drive  experiment  design  and  control. 

A  good  example  of  an  operational  thread  is  Time  Critical  Targeting  (TCT).  There  are  specific 
events  within  a  scenario  that  will  force  this  thread  to  occur,  and  there  may  be  several  initiatives 
that  contribute  to  each  segment  of  the  “kill  chain”  (Find,  Fix,  Track,  Target,  Engage,  and 
Assess).  The  contribution  of  experimental  initiatives  to  each  kill  chain  activity  may  be  measured 
in  tenns  of  timeliness,  accuracy,  completeness,  or  any  other  relevant  Measure  of  Performance 
(MOP).  Eikewise,  there  may  be  overall  Measures  of  Effectiveness  (MOE)  for  this  operational 
thread.  For  the  TCT  thread,  the  most  significant  measure  may  be  overall  time  from  finding  a 
target,  to  engagement,  and  to  assessment  of  combat  effects. 

Dr.  Richard  Kass,  in  his  paper  “Understanding  Joint  Warfighting  Experiments:  The  Logic  of 
Warfighting  Experimentation,”  argues  that  experimentation  is  uniquely  suited  to  capability 
development,  provided  that  some  basic  requirements  are  met  during  the  experiment  design 
phase.  Specifically,  Dr.  Kass  outlines  the  relative  merits  of  four  types  of  experiment:  Analytic 
Wargame,  Constructive,  Human-in-the-Loop,  and  Field.  Large-scale,  high-fidelity  field 
experiments  such  as  JEFX  are  best  suited  to  relating  results  to  operations.  However,  due  to  the 
large  number  of  uncontrolled  variables  in  such  an  environment,  it  has  typically  been  difficult  to 
isolate  and  examine  a  single  variable — such  as  the  contribution  of  a  new  technology  or  process. 
We  believe  that  OTD  has  the  potential  for  improving  the  utility  of  the  results  of  field 
experimentation  by  helping  to  identify  variables,  ensuring  that  players  and  controllers  are  fully 
aware  of  analytic  objectives  for  the  experiment,  and  providing  a  context  for  relating  the  results  to 
“real-world”  operations. 

Thread  Development  Process 

Capability  Development  Teams 

This  experiment  included  the  use  of  a  new  organizational  construct  for  planning,  managing,  and 
controlling  the  experiment.  The  four  Capability  Development  Teams  (CDTs)  were  given 
responsibility  for  achieving  each  of  the  four  Capability  Goals — CAPE,  Fusion  for  the  AOC,  CN, 
and  HLS/HLD.  The  leadership  of  these  teams  played  a  significant  role  in  developing  the 
operational  threads.  Each  CDT  also  designated  Operational  Thread  Managers  (OTMs) — one  for 
each  operational  thread  assigned  to  the  CDT — who  led  the  development  of  the  operational 
threads.  The  OTM  was  also  often  the  lead  assessor  for  experiment  initiatives  highlighted  within 
the  context  of  their  respective  operational  threads. 

AFEO  proposed  a  phased  process  for  developing  operational  threads  during  the  initial 
presentation  of  this  concept  to  the  CDTs.  At  that  time,  there  was  little  feedback  or  discussion 
regarding  this  proposal.  Over  time,  it  became  apparent  that  an  incremental  approach  to 


accomplishing  this  complex  task  was  required.  As  the  managers  of  this  process,  AFEO  staff  was 
consistently  trying  to  stay  one  step  ahead  of  those  who  were  doing  the  work.  The  basic  phased 
approach  is  depicted  in  Figure  2. 
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Figure  2 — Phased  Approach 

The  primary  5  phases  are  indicated  in  the  center  of  this  figure,  and  related  activities  are  shown  as 
well.  The  spiral  development  approach  for  JEFX  allowed  the  assessment  teams  to  manage  the 
workload  across  several  events,  while  constantly  improving  upon  experiment  design. 

Phase  1,  deficiency  identification,  was  the  primary  focus  of  the  Concept  Development 
Conference  (CDC),  held  in  November,  2004.  Some  of  this  work  had  already  been  accomplished 
prior  to  the  conference,  leaving  time  at  the  CDC  to  focus  on  further  defining  the  specific 
deficiencies  and  Measures  of  Success. 

Capability  Deficiency  Identification 

During  the  CDC,  the  four  JEFX  capability  goals  were  further  refined  by  the  CDTs  into  13 
specific  capability  deficiencies  that  would  be  the  primary  focus  of  this  experiment: 

•  CAPE  Deficiencies 

o  Continuous  theater  air  planning 

o  Enhanced  situational  awareness 

o  Effects  assessment 


o  Dynamic  execution 

•  Fusion  for  the  AOC  Deficiencies 

o  Multi-Intelligence  (INT)  fusion  for  the  kill  chain 
o  CAOC-Distributed  Common  Ground  System  (DCGS)  integration 
o  Multi-INT  fusion  for  predictive  analysis 

•  CN  Deficiencies 

o  Inadequate  enabling  of  Command  and  Control  (C2) 
o  Inadequate  enabling  of  defensive  measures 
o  Inadequate  enabling  of  Joint  Blue  Force  SA  and  Combat  ID 

•  HLS/HLD  Deficiencies 

o  Counter  Sea 
o  CONUS  Counter  Air 

o  Defense  support  to  civil  authorities  -  emergency  response 

After  the  CDC,  deficiency-specific  infonnation  was  entered  into  the  newly  developed 
operational  thread  toolset.  This  additional  information  specified  related  Operational 
Requirements  Document  (ORD)  or  Capability  Development  Document  (CDD)  defined 
requirements.  Deficiency  identification  was  the  fundamental  first  step,  and  is  critical  to  all 
phases  that  follow.  During  the  formal  JEFX  initiative  selection  process,  four  primary  criteria 
were  considered:  relevance  to  defined  capability  deficiencies,  potential  for  operational  utility, 
technical  maturity,  and  total  cost.  Immediately  following  deficiency  identification  and  the 
initiative  selection  process,  the  CDTs  began  developing  operational  threads  and  defining  the 
tasks  that  comprise  each  thread. 

Thread  Identification  and  Task  Development 

The  idea  that  operational  threads  should  or  could  be  integrated  in  some  way  posed  a  conceptual 
challenge  at  this  point  in  the  experiment  design  process.  Originally,  the  framework  had  not  been 
designed  to  account  for  such  a  relationship,  but  we  soon  realized  the  importance  of  adapting  to 
accommodate  this  requirement.  The  initial  target  for  JEFX  06  was  to  have  between  15  and  20 
operational  threads.  Once  we  began  to  develop  the  threads,  however,  some  participants 
questioned  the  rationale  for  having  any  more  than  a  single  operational  thread.  Ultimately,  we 
selected  an  existing  framework  based  on  joint  doctrine  to  provide  a  means  of  identifying  the 
relationship  among  the  operational  threads.  This  framework  allowed  us  to  avoid  having  a  single 
thread — which  would  have  been  difficult  to  manage — while  providing  an  integrating  framework 
that  clarified  the  relationship  among  the  various  threads.  The  framework  we  chose  is  the  Air 
Tasking  Cycle,  found  in  Joint  Publication  3-30,  Command  and  Control  for  Joint  Air  Operations, 
and  depicted  in  Figure  3. 


Figure  3 — Joint  Air  Tasking  Cycle 

The  CDT  effort  eventually  resulted  in  22  operational  threads  for  JEFX  06: 

•  Joint  Air  Estimate  Process  (JAEP) 

•  Force  Application 

•  Build  Master  Air  Attack  Plan  (MAAP) 

•  Air  Tasking  Order  (ATO)  Production 

•  Base  Infrastructure 

•  Enhance  C2  SA  with  Non-Traditional  Intelligence,  Surveillance  and  Reconnaissance 
(NTISR) 

•  Prosecute  NTISR 

•  Operational  Assessment 

•  Tactical  Synchronization 

•  Monitor  the  Common  Operational  Picture 

•  Special  Operations  Forces  (SOF)  Planning  &  Execution 

•  Prosecute  TST 


•  Check  weather  (WX)  for  dynamic  target 

•  IP  platform  requests  WX 

•  Near  Space  Radio  Net  Utilization 

•  Combat  Assessments 

•  Manage  the  Constellation  Net 

•  Improve  CN  Defense 

•  Maritime  Threat  (Lead  Federal  Agency) 

•  Joint  Blue  Force  Situational  Awareness 

•  Air  Mobility  Division  (AMD)  Distributed  Operations 

•  Phase  IV  Transition  Planning 

By  selecting  a  framework  based  on  joint  doctrine  and  used  by  the  warfighting  community,  it  was 
much  easier  to  communicate  the  analysis  objectives  to  the  rest  of  the  experiment  enterprise — 
including  players,  controllers,  engineers,  and  leadership — and  gain  support  for  those  objectives. 

Once  the  CDTs  and  operational  thread  managers  had  defined  the  threads  and  tasks,  the  next 
phase  was  to  develop  Measures  of  Effectiveness  (MOE)  for  the  operational  threads  and 
Measures  of  Performance  (MOP)  for  each  task. 

Measures  Development 

Developing  measures  proved  to  be  the  most  difficult  task  for  the  team.  A  majority  of  the  AIPT — 
and  many  of  the  individual  CDT  members — were  recruited  for  specific  subject  matter  expertise, 
but  were  not  necessarily  trained  analysts.  In  order  to  facilitate  the  task  of  developing  useful, 
relevant  metrics,  the  AFEO  core  team  of  analysts  chose  the  Network-Centric  Operations  (NCO) 
Conceptual  Framework  (Version  2)  as  a  source  document  for  developing  measures.  This 
document  includes  an  annex  of  metrics  that  are  related  to  many  of  the  operational  activities 
under  examination  in  this  experiment.  This  document  was  also  useful  since  NCO  was  an 
underlying  theme  for  JEFX  06. 

Training  the  analysis  team  on  the  approach  to  developing  operational  threads  and  the  use  of  the 
toolset  was  critical  to  the  success  of  this  endeavor.  Throughout  the  process,  the  AFEO  core  team 
would  provide  training  that  was  relevant  to  the  particular  phase  of  thread  development  that  most 
teams  were  involved  in.  During  the  development  of  measures,  the  AFEO  core  team  assisted  each 
CDT  in  developing  assessment  plans  that  outlined  the  thread  MOE  and  described  the 
contribution  of  initiatives  using  MOP  for  each  thread  task.  We  also  gained  support  from  a 
separate  team  within  the  AFC2ISRC  that  was  familiar  with  the  theory  of  net-centric  warfare  and 
the  concept  for  developing  net-centric  systems  and  processes. 

The  measures  that  were  developed  during  this  phase  formed  the  basis  for  the  most  tedious  phase 
of  operational  thread  development — definition  of  data  collection  requirements. 


Data  Collection  Requirements 

As  deficiency  identification  is  critical  and  measure  development  is  the  most  difficult  phase  of 
OTD,  the  development  of  a  single,  integrated  list  of  Data  Collection  Requirements  (DCRs)  is  the 
most  tedious  and  coordination-intensive  phase.  The  DCRs  provided  the  “who,  what,  when, 
where,  and  how”  planning  details  for  obtaining  the  data  necessary  to  compute  the  analysis 
measures.  Development  of  DCRs  served  several  purposes.  First,  development  of  a  shared  set  of 
collection  requirements  prevented  duplication  of  effort.  This  shared  view  of  the  DCRs  was 
possible  primarily  through  the  use  of  a  web-based  toolset.  In  addition,  detailed  DCRs  provided  a 
tremendous  coordination  tool  to  distribute  collection  activities  among  a  limited  number  of  data 
collectors.  Data  collection  activities  were  synchronized  with  the  schedule  for  execution  of  the 
operational  threads.  Finally,  the  data  collection  requirements  for  large-scale  experiments  were 
historically  fragmented,  and  not  well-connected  to  the  overall  experiment  objectives.  The 
analysis  approach  employed  for  this  experiment,  and  the  supporting  toolset  that  was  developed  to 
capture  related  infonnation,  provided  a  framework  for  the  analysis  team  to  document  all  of  the 
required  data. 

For  this  experiment,  the  Joint  Fires  Interoperability  and  Integration  Team  (JFIIT) — a  sub¬ 
organization  of  Joint  Forces  Command  (JFCOM)  J-8 — led  the  data  collection  effort.  The 
operational  thread  construct  and  supporting  toolset  provided  a  valuable  method  for 
communicating  analysis  objectives  to  the  data  collectors,  as  well  as  for  establishing  specific  data 
collection  requirements. 

During  the  experiment  planning  phase,  and  also  during  the  execution  of  spiral  events,  the  JFIIT 
team  met  with  each  CDT  and  operational  thread  manager  to  refine  the  measures  and  associated 
data  collection  requirements.  This  interaction  was  critical  to  achieving  a  common  understanding 
among  analysts  and  data  collectors.  In  addition,  having  three  separate  spiral  events  prior  to  the 
main  experiment  allowed  the  assessment  team  to  continually  refine  tools  and  processes  in 
preparation  for  the  final  event. 

Thread  Execution 

During  Spiral  2  for  JEFX,  the  assessment  team  had  the  first  opportunity  to  see  the  operational 
threads  executed  and  refine  the  data  collection  process.  A  daily  battle  rhythm  emerged  that 
included  a  daily  review  of  objectives,  specification  of  the  “threads  of  the  day,”  data  collection 
activity,  and  end  of  day  review  and  analysis.  In  addition,  several  ideas  were  generated  for 
improving  the  toolset,  and  several  new  features  were  implemented  prior  to  the  next  spiral  event. 

Web-Based  Toolset 

The  development  of  a  web-based  toolset  that  was  incorporated  into  the  existing  experiment 
management  system  greatly  improved  operational  thread  development.  This  toolset  was 
developed  in  conjunction  with  the  conceptual  framework  for  operational  threads  and,  as  a  result, 
supports  that  framework. 

Toolset  Development 

The  OTD  system  was  designed  around  a  Microsoft  Structured  Query  Language  (SQL)  Server 
2000  back  end  and  web-enabled  front  end  with  Microsoft  Internet  Information  Server  as  the  host. 


The  toolset  uses  Active  Server  Pages  (ASP),  Hypertext  Markup  Language  (HTML),  and 
JavaScript. 

Prototype  toolset  development  began  in  June  2005  using  the  Rapid  Application  Design 
Methodology  (RAD).  To  the  extent  possible,  requirements  were  gathered  based  on  all 
information  available  at  that  time.  This  prototype  and  the  RAD  development  methodology 
served  to  point  out  initial  flaws  in  the  OTD  process,  but  were  eventually  determined  to  be 
unworkable. 

The  next  toolset  build  used  an  alternative  approach  known  as  the  Agile  methodology.  Based  on 
the  extensive  changes  to  data  relationships  and  an  evolving  OTD  process,  we  believed  this 
approach  would  yield  improved  results.  The  Manifesto  for  Agile  Software  Development 
describes  the  following  tenets: 

•  Individuals  and  interactions  over  processes  and  tools 

•  Working  software  over  comprehensive  documentation 

•  Customer  collaboration  over  contract  negotiation 

•  Responding  to  change  over  following  a  plan 

This  methodology  worked  out  much  better  and  over  the  life  of  the  project  has  afforded  far  more 
flexibility  in  building  the  toolset  to  support  the  OTD  process  that  was  being  concurrently 
developed. 

The  lack  of  incremental  testing  opportunities  was  a  critical  challenge  in  the  development  of  this 
toolset.  Because  users  of  the  toolset  were  investing  significant  time  in  populating  the  database 
with  thread  information,  we  were  seldom  able  to  conduct  preferred  levels  of  toolset  testing 
before  fielding  new  increments.  Over  time,  the  users  became  dependent  on  the  toolset  and, 
therefore,  any  system  problems  could  have  negatively  affected  the  ongoing  OTD  process. 

Spiral  2  was  the  first  extensive  field  test  of  the  usefulness  of  the  OTD  toolset.  The  system 
worked  without  any  major  problems.  Most  fixes  were  made  immediately  and  no  OTD  toolset 
work  stoppages  were  encountered.  After  this  event,  as  the  OTD  process  continued  to  progress, 
new  functionality  was  added  to  the  toolset  and  testing  continued  during  Spiral  3. 

In  preparation  for  the  main  experiment,  the  remaining  required  features  of  the  toolset  are  being 
implemented.  By  providing  assessment  results  pertinent  to  capability  deficiencies  and 
experiment  initiatives,  this  new  functionality  will  assist  lead  assessors  in  creating  the  experiment 
final  report. 

Toolset  Security 

The  OTD  system  was  developed  within  the  security  framework  of  the  existing  AFEO  Webtools 
Portal  so  all  user  information  and  privileges  are  handled  by  the  security  functionality  of  the 
Portal.  Only  one  new  security  group  was  created  to  support  the  OTD  toolset:  the  Thread 
Administrators  Group. 

Security  Roles: 

•  Thread  Administrators:  have  overall  privileges  throughout  the  entire  OTD  toolset. 


•  Operational  Thread  Managers  (OTMs):  have  full  control  over  only  the  threads  they  have 
been  assigned  to  as  an  OTM.  OTMs  can  create  and  edit  all  required  objects  (e.g.;  MOE, 
tasks,  MOP)  and  provide  assessments  of  the  measures 

•  Thread  Editors:  have  the  same  privileges  as  OTMs,  except  they  cannot  provide 
assessments  of  measures 

Portal  users  with  Assessor  privileges  can  view  all  OTD  information  and  provide  comments  for 
later  assessment.  All  Portal  users  with  general  access  can  view  the  basic  OTD  data,  although 
they  have  no  access  to  assessment  information. 

Figure  4  depicts  the  basic  relationship  among  the  following  database  entities: 

Threads:  Operational  threads,  as  described  in  this  paper. 

Tasks:  The  basic  component  of  an  operational  thread. 

Measures  of  Effectiveness  (MOE):  Characterize  the  overall  effectiveness  of  a  thread. 

Deficiencies:  Capability  deficiencies,  as  described  in  the  Analysis  Framework. 

Measures  of  Performance  MOP):  Characterize  the  perfonnance  of  a  single  task. 

Initiatives:  New  technology,  processes,  or  organizational  structures  that  contribute  to  a 
capability  deficiency. 

Data  Collection  Requirement  (DCR):  The  information  that  must  be  collected  in  order  to 
compute  an  MOP. 

Additional  information  regarding  the  structure  of  the  toolset  is  found  in  Appendix  3. 


Figure  4 — Basic  Database  Layout 


Challenges 

During  the  development  of  this  toolset,  we  encountered  and  overcame  many  challenges: 

•  Overcoming  resistance  to  change  among  those  who  had  been  involved  in  experimentation 
for  many  years 

•  Accommodating  re-definition  of  key  terms  and  relationships,  based  on  new  insights 
during  the  planning  process 

•  Balancing  the  desire  to  add  new  functionality  with  the  goal  of  keeping  the  toolset  as 
intuitive  and  “user-friendly”  as  possible 

•  Accommodating  the  requirements  of  a  large  user  base 

•  Understanding  a  process  that  was  in  development,  and  developing  software  to  support 
that  emerging  process 

•  Keeping  the  requirements  within  reach  of  what  could  be  accomplished  in  the  time  given 


Successfully  overcoming  these  challenges  was  the  result  of  having  a  developer  onsite  that  is 
familiar  with  experimentation.  Based  on  daily  coordination  between  the  developer  and  the  users 
of  the  OTD  process,  we  were  able  to  realize  many  benefits  of  having  such  a  tool. 

Benefits 

The  benefits  of  using  a  web-based  toolset  were  numerous: 

•  Insight  into  the  progress  of  thread  development 

•  A  single  source  for  operational  thread  information,  with  no  version  control  confusion 

•  Accessibility  of  information  via  a  web  browser  and  internet  connection 

•  Linkage  of  all  planning  details,  in  context  of  the  analysis  framework 

•  Linkage  of  all  results,  in  context  of  the  analysis  and  reporting  framework 

•  Forced  adherence  to  the  model — something  spreadsheets  do  not 

•  Shared  view  among  entire  experiment  enterprise 

Finally,  there  are  several  enhancements  that  we  would  like  to  incorporate  for  the  next  major 
experiment. 

Future  Enhancements 

•  Development  of  a  thick  client  version 

•  Further  development  of  the  reporting  functionality 

A  depiction  of  the  front  page  of  the  toolset  is  shown  in  Figure  5. 
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Figure  5 — Online  Operational  Thread  Toolset 

As  indicated  in  Figure  5,  this  toolset  captures  and  displays  relevant  infonnation  about  an 
operational  thread.  The  initial  view  provides  basic  information,  such  as  the  number  and  title  of 
the  operational  thread,  the  thread  manager,  linkage  to  capability  deficiencies  and  experimental 
initiatives,  and  related  files.  A  more  detailed  view — as  shown  in  Figure  6 — shows  additional 
detail  about  a  thread,  such  as  Measures  of  Effectiveness  (MOE)  and  the  sequence  of  thread  tasks. 


Figure  6 — Detailed  Thread  View 

For  the  planning,  design,  assessment  and  reporting  of  JEFX  06,  this  toolset  was  used  extensively 
by  the  assessment  team,  experiment  controllers,  players,  systems  engineers,  live  fly  planners,  and 
others  to  capture  and  view  important  information. 

Conclusion 

Analyzing  capabilities  and  assessing  the  contribution  of  new  technology,  processes  and 
organizational  structures  poses  a  unique  challenge  for  those  involved  in  military 
experimentation.  For  practitioners  of  large-scale  field  experiments,  conducting  a  rigorous 
analysis — one  that  could  support  acquisition  decisions — is  particularly  vexing.  Over  the  course 
of  several  years  of  conducting  large-scale  experiments,  the  Air  Force  Experimentation  Office  has 
developed  a  repeatable,  rigorous  process  for  experimentation  and  a  supporting  toolset  for 
operational  thread  development  that  may  be  useful  for  others  engaged  in  joint  or  service 
experimentation.  Based  on  our  experience  during  the  planning,  execution  and  reporting  phases  of 
Joint  Expeditionary  Force  Experiment  2006,  we  believe  that  the  development  of  operational 
threads  improves  the  task  of  analyzing  warfighting  capability,  ultimately  leading  to  improved 
capability  for  the  warfighter. 


Appendix  1 — Glossary  of  Terms 

Analysis:  Analysis  involves  decomposition  of  an  area  of  examination  into  constituent  parts  for 
further  study.  In  the  context  of  this  experiment,  analysis  activities  involve  in-depth  examination 
of  narrow  areas  of  interest  such  as  technical  components  of  an  initiative,  specific  operational 
processes,  or  specific  areas  such  as  communications  or  network  architectures. 

Assessment:  For  the  purposes  of  JEFX,  “assessment”  is  the  broadest  term  that  defines  all  the 
activities  of  the  Assessment  Integrated  Product  Team  (AIPT.)  Assessment  is  broader  than 
“analysis”  in  that  assessment  is  an  activity  that  involves  other  experiment  participants  (e.g., 
operators)  whereas  analysis  is  primarily  an  activity  for  the  AIPT.  In  addition,  whereas  analysis  is 
primarily  a  decomposition  activity,  assessment  is  a  process  of  synthesizing  information  to 
provide  an  overall  appraisal  of  a  broad  area  of  examination.  In  the  context  of  JEFX,  assessment 
implies  more  than  strictly  placing  value  on  a  new  technology,  concept,  or  idea.  Assessment  in 
this  context  also  involves  collecting  relevant  infonnation  with  the  goal  of  providing  an  unbiased 
explanation  of  how  a  new  concept  or  technology  could  integrate  into  an  operational  level  C2 
architecture. 

Capability:  The  ability  to  achieve  a  desired  effect  under  specified  standards  and  conditions 
through  combinations  of  means  and  ways  to  perfonn  a  set  of  tasks  (CJCSI  3 170.01E,  Joint 
Capabilities  Integration  and  Development  System).  Inherent  to  a  capability  are  the  organizations 
and  people,  processes,  and  technical  means  used  to  accomplish  a  military  task  or  mission. 
Standard  US  Air  Force  capabilities  are  found  in  the  Master  Capability  Library. 

Capability  Assessment:  The  capability  assessment  strives  to  determine  how  well  the  experiment 
addressed  a  particular  capability  deficiency,  through  the  use  of  initiatives  that  supported 
operational  activities  (such  as  Crisis  Action  Planning.)  The  capability  assessment  complements 
the  initiative  assessment  by  examining  areas  outside  the  assessment  of  individual  initiatives,  and 
also  putting  initiatives  into  the  context  of  a  broader  joint  warfighting  capability. 

Capability  Gap:  The  inability  to  achieve  a  desired  effect  under  specified  standards  and 
conditions  through  combinations  of  means  and  ways  to  perfonn  a  set  of  tasks  (CJCSI  3 170.01E). 
For  JEFX,  this  term  is  synonymous  with  capability  “deficiency”.  Capability  gaps  are  chosen  by 
CDTs  based  on  the  experiment  focus  areas  and  capability  goal  statements.  CDTs  must  be 
selective  in  choosing  capability  gaps;  time  and  resource  limitations  often  prevent  us  from 
achieving  all  aspects  of  a  broadly  stated  capability  goal 

Capability  Goal:  A  statement  of  intent,  formulated  by  the  respective  CDT  and  based  on 
experiment  focus  areas,  that  defines  an  end  state  for  correcting  deficiencies  and  closing  one  or 
more  capability  deficiencies  during  the  course  of  the  experiment. 

Data  Analysis  Cell:  A  core  team  provided  by  AFEO  to  support  the  data  analysis  effort  for 
JEFX.  This  group  of  analysts  will  review  inputs  from  Jefxlink,  produce  summary  results,  and 
recommend  additional  data  collection. 

Data  Collection  Cell:  A  core  team  comprised  of  the  Joint  Fires  Interoperability  and  Integration 
Team  (JFIIT),  605  Test  and  Evaluation  Squadron  and  the  505  Operations  Squadron  collectively 
responsible  for  planning  and  supporting  the  data  collection  effort.  DCC  responsibilities  include 
collecting  data  from  all  required  sources  and  facilitating  a  daily  phased  debrief  for  C2  and  live 
fly  players. 


Demonstration:  May  or  may  not  be  connected  to  the  experiment  systems  infrastructure  and 
operational  processes.  Demonstrations  will  consume  very  limited  or  no  experiment  design 
resources,  and  there  will  be  no  experiment  enterprise-sanctioned  assessment  beyond  a  brief 
description  of  the  demonstration  itself. 

DOTMLPF:  Doctrine,  Organization,  Training,  Materiel,  Leadership,  Personnel  and  Facilities. 
These  broad  categories  provide  a  useful  framework  for  organizing  results  or  recommendations. 

In  some  cases,  JEFX  results  will  provide  additional  infonnation  for  inclusion  into  a  formal 
DOTMLPF  change  recommendation  package,  as  specified  by  the  Joint  Capability  Integration 
and  Development  System  (JCIDS). 

Initiative:  For  the  purposes  of  JEFX,  an  “initiative”  is  any  specific  equipment,  concept,  process 
or  new  technology  that  has  been  officially  selected  by  the  JEFX  competitive  selection  process, 
culminating  in  approval  by  CSAF.  Approximately  65%  of  the  initiatives  are  required  to  have  the 
capability  for  fielding  within  18  months  of  CSAF  approval.  Initiatives  are  high-visibility 
elements  of  the  experiment  and  will  likely  consume  the  majority  of  design  and  assessment 
resources. 

Initiative  Core  Capability:  The  functions  that  an  initiative  provides  to  the  warfighter,  based  on 
input  from  the  sponsor.  These  are  not  to  be  confused  with  the  capabilities  described  in  an  AF 
CONOPS,  which  are  often  higher-level. 

Initiative  Provider:  The  contractor  or  agency  responsible  for  developing,  providing,  and 
supporting  the  initiative  during  JEFX. 

Initiative  Sponsor:  The  government  agency  responsible  for  submitting  the  initiative  for 
selection,  coordinating  all  support  for  the  initiative  during  planning  and  execution  of  the 
experiment,  and  coordinating  transition  of  the  initiative. 

Item  of  Interest  (IOI):  A  technology,  process  or  system  that  is  included  in  the  experiment 
architecture  that  facilitates  the  experiment  and  requires  experiment  management  or  leadership 
tracking  and/or  reporting. 

Key  Enabler:  As  part  of  the  infrastructure,  a  Key  Enabler  is  any  technology,  process, 
organizational  structure,  or  idea  that  has  not  been  officially  selected  by  the  JEFX  competitive 
initiative  selection  process.  Key  Enablers  are  subject  to  the  formal  Configuration  Control  Board 
(CCB)  process,  and  must  be  approved  through  that  mechanism  for  inclusion  in  the  experiment. 
Key  Enablers  are  not  eligible  for  JEFX  program  transition  funding,  and  will  generally  be 
secondary  to  initiatives  in  terms  of  access  to  experiment  design  and  assessment  resources.  Key 
Enablers  may  be  pre-identified  or  may  occur  at  any  time  during  the  spirals  or  main  experiment, 
subject  to  the  CCB  process.  Because  this  experiment  is  designed  to  foster  innovation,  Key 
Enablers  will  typically  be  assessed  by  the  assessment  team  in  terms  of  contribution  to  enabling 
successful  demonstration  of  warfighter  capability  goals  or  as  part  of  a  solution  to  a  warfighter 
capability  deficiency.  However,  relative  to  initiatives,  they  will  have  a  lower  priority  of  access  to 
resources.  Key  Enablers  will  typically  be  self- funded  to  include  providing  resources  for 
assessment  and  other  JEFX  enterprise  efforts. 

Measure  of  Effectiveness:  According  to  the  AF  Analyst’s  Handbook  (produced  by  the  Office  of 
Aerospace  Studies),  MOE,  when  evaluated,  “quantitatively — and  occasionally  qualitatively — 
describe  how  well  tasks  are  performed.”  For  C2  assessment  activities  (such  as  JEFX)  the  tenn 
“task”  may  be  taken  in  a  broader  sense  to  mean  “a  series  of  related  tasks” — a  process.  With  this 


broadening  of  the  standard  MOE  definition,  JEFX  assessment  recognizes  the  importance  of 
process  to  command  and  control.  The  NATO  Code  of  Best  Practices  for  C2  Assessment  defines 
a  “Measure  of  C2  Effectiveness”  that  focuses  on  the  impact  of  C2  systems  and  processes  within 
an  operational  context.  This  definition  more  closely  applies  to  what  we  are  doing  in  JEFX. 

Measure  of  Performance:  In  the  context  of  JEFX  assessment,  an  MOP  characterizes  the 
performance  of  a  single  task.  Unlike  an  MOE,  it  does  not  indicate  the  effectiveness  of  that  task, 
or  set  of  related  tasks,  but  instead  describes  a  very  specific  characteristic,  such  as  the  time  to 
complete  a  specific  task  or  the  man-hours  involved.  Using  the  F2T2EA  model,  an  MOP  might  be 
the  number  of  sensors  involved  in  accurately  fixing  an  emerging  target,  or  the  time  to  complete 
the  “Fix”  step  in  the  process. 

Measure  of  Success:  Conceptually  the  highest  level  measure  within  the  analysis  planning 
framework,  MOS  characterize  the  overall  success  in  achieving  a  particular  capability  goal. 

Operational  Thread:  An  “operational  thread”  is  a  series  of  related  operational  tasks  that  are 
specifically  focused  to  highlight  the  contribution  of  experimental  initiatives  or  infrastructure 
systems  to  an  Air  and  Space  Operations  Center  (AOC)  or  other  basic  Command  and  Control 
(C2)  process.  An  operational  thread  is  a  design  feature  of  the  experiment,  allowing  experiment 
planners  to  influence  player  activity  in  desired  areas.  These  threads  generally  have  a  well- 
defined  beginning  and  end,  and  are  often  executed  within  a  single  period  during  the  experiment. 
Operational  threads  are  typically  stimulated  by  scenario  events  (from  the  Master  Scenario  Event 
List — MSEL)  but  they  may  involve  a  significant  degree  of  “free  play”  by  the  players. 

Operational  threads  are  often  related  to  AOC  processes,  but  they  are  unique  to  an  experiment. 
Because  initiatives  sometimes  involve  activities  that  are  different  from  current  practice, 
operational  threads  can  force  an  examination  of  those  activities  in  a  new  context.  Every 
operational  thread  should  be  associated  with  relevant  measures  of  effectiveness  and  perfonnance. 
These  measures  characterize  the  contribution  of  initiatives  to  the  operational  activity,  and  also 
indicate  the  overall  effectiveness  of  the  underlying  AOC  process. 

Other  Service  Initiative:  For  the  purposes  of  JEFX,  an  “other  service  initiative”  is  any  specific 
equipment,  concept,  process  or  new  technology  that  has  been  vetted  through  the  JEFX  enterprise. 
Other  service  initiatives  are  not  eligible  for  JEFX  program  transition  funding.  Services  are 
responsible  for  design,  funding,  assessment  and  transition.  There  may  be  some  integration  with 
JEFX,  but  they  will  not  interfere  with  the  experiment  and  may  enhance  experiment  established 
threads. 
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Appendix  3 — Web-Based  Toolset 


The  database  contains  the  following  data  tables  to  support  the  toolset: 

Table  1 — Data  Tables 


Data  Table 

Description 

Threads 

This  is  the  primary  table  holding  the  basic  information  about 
each  thread  and  establishing  the  ThreadID  that  will  be  used 
as  the  foreign  key  (FKey)  throughout  the  system. 

MOE 

Tied  to  a  single  Thread  by  the  FKey:  ThreadID. 

Tied  to  multiple  Tasks  via  the  MOETaskLink  table 

Tasks 

Tied  to  a  single  Thread  by  the  FKey:  ThreadID. 

Tied  to  multiple  MOEs  via  the  MOETaskLink  table 

Tied  to  Deficiencies  by  the  FKey:  GapID 

Tied  to  Initiatives  by  the  FKey:  InitID. 

Deficiencies 

Tied  to  Initiatives  by  the  FKey:  ThreadID. 

MOP 

Tied  to  a  single  Thread  by  the  FKey:  ThreadID. 

DCR 

Tied  to  a  single  MOP  by  the  FKey:  MOPID. 

Initiatives 

Holds  information  about  the  specific  initiatives  linked  to  the 
Tasks. 

Documents 

Holds  information  about  the  documents  that  are  linked  to  the 
Threads  and  Deficiencies 

Personnel 

Holds  information  about  users  including  permissions. 

The  supporting  (normalized)  tables  are  used  to  provide  additional  data  about  the  object 
entities  they  support: 


Table  2 — Normalized  Tables 


Table  Name 

Description 

ThreadGrp 

Threads  are  assigned  to  Groups  to  for  the  sake  of  sorting  and 
display.  This  table  holds  the  infonnation  about  each  group. 

ThreadPriv 

Holds  the  security  information  about  each  Thread  to  include 
Owner  and  Editors  personnel  record  ID  numbers. 

TaskEval 

Holds  the  assessment/evaluation  information  about  each 

Task.  Tasks  are  assessed  during  each  event  so  separate 
records  are  needed  for  each  event. 

DCREval 

Holds  the  assessment/evaluation  information  about  each 

DCR.  DCRs  are  assessed  during  each  event  so  separate 
records  are  needed  for  each  event. 

DCRSub 

Allows  users  to  subscribe  to  a  specific  DCR  and  be  notified 
of  changes. 

ThreadExecSch 

Holds  the  execution  schedules  (times)  for  each  thread. 

The  linking  tables  are  used  to  establish  “many  to  many”  table  relationships: 


Table  3 — Linking  Tables 


Table  Name 

Description 

ThreadDocLink 

Links  Threads  to  Documents. 

MOETaskLink 

Links  MOEs  to  Tasks. 

GapDocLink 

Links  Deficiencies  to  Documents. 

Appendix  4 — Acronyms 


A 

ACASSA 

ACS 

AFC2ISRC 

AIPT. 

AMD 

AOC 

ASP 

ATO 

C 

C2 

CAPE 

CDC 

CDD 

CDTs 

CN 

D 

DCGS 

DCR 

H 

HLS/HLD 

I 

ICCRTS 

INT 

IOI 

IP 

J 

JAEP 
JEFX  06 
JFCOM 


Air  Support  and  Situational  Awareness 
Agile  Combat  Support 

Air  Force  Command  and  Control  and  Intelligence,  Surveillance,  and 
Reconnaissance  Center 

Assessment  Integrated  Product  Team 

Air  Mobility  Division 

Air  and  Space  Operations  Center 

Active  Server  Pages 

Air  Tasking  Order 

Command  and  Control 

Continuous  Theater  Air  Planning  and  Dynamic  Execution 
Concept  Development  Conference 
Capability  Development  Document 
Capability  Development  Teams 
Constellation  Net 

Distributed  Common  Ground  System 
Data  Collection  Requirement 

Homeland  Security  /  Homeland  Defense 

International  Command  and  Control,  Research  and  Technology 
Symposium 

Intelligence 

Item  of  Interest 

Internet  Protocol 

Joint  Air  Estimate  Process 

Joint  Expeditionary  Force  Experiment  2006 

Joint  Forces  Command 


JFIIT 

M 

MAAP 

MOE 

MOP 

MOS 

MSEL 

N 

NCO 

NTISR 

O 

ORD 

OTD 

R 

RAD 

S 

SOF 

T 

TST 

W 

wx 


Joint  Fires  Interoperability  and  Integration  Team 

Master  Air  Attack  Plan 
Measure  of  Effectiveness 
Measure  of  Performance 
Measure  of  Success 
Master  Scenario  Event  List 

Network-Centric  Operations 

Non-Traditional  Intelligence,  Surveillance  and  Reconnaisance 

Operational  Requirements  Document 
Operational  Thread  Development 

Rapid  Application  Design  Methodology 

Special  Operations  Forces 

Time  Sensitive  Targeting 

Weather 
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■  Background 

■  Analysis  Framework 

■  Operational  Thread  Development  Process 

■  Online  Toolset 
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Background 

Benefit 
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■  Standardized  approach 

■  Facilitates  cross-team  collaboration 

■  Consistent  terminology  usage 

■  Improves  experiment  design 

■  Supports  joint  and  service  capability-based  assessment 
framework 

■  Analysis  support 

■  Core  analysts  will  assist  in  application  of  this  approach 

■  Phased  approach 

■  Manageable  workload  for  CDTs 

■  Clearly  identified  milestones  for  thread  development 
products 
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Background 

Purpose  of  Operational  Threads 
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■  Facilitate  examining  a  potential  improvement  to  a  deficient 
capability 

■  The  contribution  of  one  or  more  initiatives  or  improved 
infrastructure  either  through 

■  a  new  process  or  modification  to  an  existing  process 

■  a  new  organizational  construct 

■  a  new  system  level  (i.e.,  “machine-to-machine”)  dialogue  between  databases, 
applications,  or  hardware 

■  Allow  us  to  influence  player  activity  (by  tailoring  scenario 
events)  to  ensure  we  are  able  to  demonstrate  capability  goals 

■  Provide  operational  context  and  therefore  relevance 

■  When  reporting  results  (of  initiatives,  capability  goals,  anything 
else) 

Identify  the  contribution  of  initiatives  to  operationally  significant  activities 

and  processes  (i.e.,  operational  threads) 


Integrity  -  Service  -  Excellence 


U.S.AIR  FORCE 


Analysis  Framework 

Basic  Definitions 


■  Capability:  The  ability  to  achieve  a  desired  effect  under  specified 
standards  and  conditions  through  combinations  of  means  and  ways  to 

perform  a  set  of  tasks  (CJCSI  31 70.01  E)  Inherent  to  a  capability  are  the 
organizations  and  people,  processes  and  technical  means  used  to 
accomplish  a  military  task  or  mission.  Standard  AF  capabilities  are  found 
in  the  Master  Capability  Library  Version  5.5. 

■  Task:  A  discrete  event  or  action — not  specific  to  a  single  unit,  weapon 
system,  or  individual — that  enables  a  mission  or  function  to  be 
accomplished — by  individuals  or  organizations  (AF  Doctrine  Center 
glossary.)  Standard  C2  tasks  are  found  in  the  C2  Task  List  developed  by 
the  C2  Capability  Assessment  Team. 

■  Initiative:  Any  potential  solution — across  the  DOTMLPF  spectrum — for 
addressing  a  recognized  warfighting  need  or  capability  gap.  Initiatives 
result  from  the  JEFX  initiative  selection  process. 

■  Capability  Gaps:  The  inability  to  achieve  a  desired  effect  under  specified 
standards  and  conditions  through  combinations  of  means  and  ways  to 
perform  a  set  of  tasks  (CJCSI  31 70.01  E).  Synonymous  with  capability 
“deficiency” 
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JEFX  Analysis  Planning 
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Measure 
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Measures  focus  on  operational 
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performance;  selected  on  the 
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Who,  What 
When,  Where 
How,  Format 
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Operational  Thread  Definition 
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■  Definition:  A  series  of  operational  tasks  that  relate 
initiatives  and/or  improved  infrastructure  systems  to 
one  or  more  C2  processes 

■  Characteristics 

■  A  design  feature  of  the  experiment;  used  by  Capability 
Development  Teams  to  assess  contribution  of  potential 
solutions  to  capability  gaps 

■  Should  be  represented  by  an  operational  architecture  and 
supporting  systems  views;  facilitates  transition  within  the 
Joint  Capabilities  Integration  and  Development  System 
(JCIDS)  (OV-6C) 

■  Observable  and  measurable;  defines  specifically  what  the 
assessment  team  will  examine 
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Analysis  Framework 

Required  Elements 
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■  Name  &  Identifier:  Uniquely  identifies  the  thread  (e.g., 
“01  A:  Joint  Air  Estimate  Process”) 

■  Description:  Description  of  the  capability  deficiency 
or  gap  this  operational  thread  will  examine 

■  Operational  Tasks:  Related  C2  &  ISR  tasks  and  player 
activities 

■  Measures:  Characterize  the  performance  of  tasks  and 
overall  effectiveness  of  the  operational  thread 

■  Initiatives:  Initiatives  (and  innovations)  that  contribute 
to  the  thread  (i.e.,  potential  “solutions”) 
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■  Provides  a  framework  to  view  the 
interrelationship  among  operational  threads 

■  Incorporates  most  operational  threads,  but  not 
everything 

■  Homeland  Security  threads  do  not  fit  this  construct 

■  Some  Constellation  Net  specific  threads  do  not  fit 

■  Possibly  others  that  do  not  fit  (e.g.,  SOF  specific) 

■  Based  on  Joint  Publication  3-30,  Command  and 
Control  for  Joint  Air  Operations 

■  Air  Tasking  Cycle 
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Development  Process 

Joint  Air  Tasking  Cycle 
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Figure  111-13.  Joint  Air  Tasking  Cycle 
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Phased  Approach 
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Development  Process 

Gap  Identification 


■  Identify  the  specific  capability  gap  or  deficiency  in  sufficient 
detail  to  ... 

■  Facilitate  call  for  initiatives 

■  Determine  measures  of  success  that  are  associated  with  each  gap 

■  Focus  of  CDC 

■  Should  be  complete;  reflected  in  the  sub-capability  goal  statements 
and  documentation 

■  Adding  or  revising  capability  gaps  at  this  point  affects  all  follow-on 
activities 

■  Example 

■  There  is  currently  a  deficiency  in  our  ability  to  receive  real-time 
data  from  organizations  external  to  the  AOC  and  perform  Level  0 
fusion  on  that  data. 
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Development  Process 

Task  Development 


■  Identify  a  series  of  operational  tasks  (i.e.,  an  operational  thread) 
that  allow  examination  of  each  capability  gap 

■  Begin  at  a  high  level  (e.g.,  1.  Find,  2.  Fix,  3.  Track,  4.  Target,  5. 
Engage,  6.  Assess) 

■  Add  details  and  supporting  activities  over  time,  as  required 

■  Player-operator  participation  is  essential 

■  Reference  the  Master  Capabilities  Library,  Functional  Area 
Assessment,  AOC  functional  decomposition,  AFOTTP, 
functional  area  CONOPS  &  CONEMPS 

■  Most  difficult  step — but  the  most  critical 

■  Example 

1.  Remote  sensors  collect  intelligence 

2.  AOC  ISR-D  receives  INT  data  from  remote  nodes 

3.  ISR-D  performs  Level  0  fusion 

4.  ISR-D  provides  fused  INT  product  to  end  user  (plans  &  ops) 

5.  End  user  acts  on  information 
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Development  Process 

Measure  Development 
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■  Identify  measures  for  each  task 

■  Measures  of  performance  for  each  task  and  measures  of  effectiveness 
for  each  operational  thread 

■  Refine  measures  of  success 

■  Developed  at  CDC 

■  Characterize  success  in  achieving  capability  goals 

■  Example  (for  Task  2 — receive  INT  data) 

■  Timeliness  of  receipt  of  INT  information  (time  lag  from  send  to  receive) 

■  Diversity  of  INT  received  (#  and  type  of  sources  providing  INT  data) 

■  Diversity  of  nodes  providing  INT  (#  and  location  of  remote  nodes) 
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Identify  Initiative  Contribution 
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■  Identify  the  contribution  each  initiative  will  make  to 
the  operational  tasks 

■  Could  lead  to  additional  measures 

■  Initiatives  may  contribute  to  many  operational  threads  and 
associated  tasks 

■  Coordinate  with  46TS  Det  1 

■  Lead  for  technical  assessment  of  initiatives 

■  Develop  “initiative  core  capabilities”  for  each  initiative 

■  Example  (for  Task  2 — receive  INT  data) 

■  Initiative  X  will  improve  the  timeliness  of  receipt  of  INT  from  remote 
nodes  through  improved  M2M  connection 
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Development  Process 

Data  Collection  Requirements 
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■  Identify  specific  data  collection  requirements 

■  Based  on  pre-identified  measures  (MOE  and  MOP) 

■  Collectively,  represents  an  Integrated  Data  Requirements  List  (IDRL) 

■  Example 
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Development  Process 

Consider  a  tions 


■  Planning  or  execution?  (Temporal  focus) 

■  Inter-theater  or  intra-theater?  (Geographic  focus) 

■  Where  do  initiatives  fit  into  this  thread? 

■  How  do  they  support  operational  tasks? 

■  What  measures  are  relevant? 

■  Based  on  Network  Centric  Operations  Conceptual  Framework 
Version  2.0 

■  What  scenario  supports  this  thread? 

■  Real-world  or  fictitious? 

■  What  scenario  vignettes  are  required? 

■  Operational  threads  should  be  prioritized 

■  Expectation:  15-20  operational  threads  total 
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Backups 
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